feat: meter coding-agent spend and cap it per hour - #184
Merged
Conversation
PatchDeck could not say how much paid agent work it had done. `agent_runs` is a per-babysit-session record with a `pr_id` foreign key, so issue work, CI and deployment healing, release notes, social posts, and PR questions were invisible, and PR-scoped history was cascade-deleted with the PR. Adds a separate `agent_invocations` ledger — one row per `codex`/`claude` process spawn, with duration, exit code, resolved agent, model, and outcome, and no foreign keys so history outlives its target. Every spawn already funnelled through `runAgentCommand`, so metering hooks in there; an AsyncLocalStorage context established by each unit of work attributes the row. Adds `maxAgentInvocationsPerHour` (default 0, unlimited). When the rolling-hour ceiling is reached the dispatcher stops claiming agent-invoking job kinds, and any spawn started inside an already-running job — CI healing, agent fallback, conflict repair — is refused. Queued work is not failed: it waits and resumes as the window rolls. The refusal classifies as transient so it never consumes a paid retry attempt. Health-check probes are recorded but never counted, so opening Settings cannot exhaust a budget. Ledger rows are pruned after 30 days on the existing retention sweep, and rows left `running` by a hard shutdown are closed at boot. Surfaced through `GET /api/agent-spend`, the `get_agent_spend` MCP tool, a header pill shown once a ceiling is set, and a Settings field. Plan: docs/plans/agent-spend-metering.md Verified with: npm run check, npx eslint ., npm run build, npm run test:all (827)
Contributor
✅ Documentation UpdatedThis PR includes documentation changes: The docs build succeeded. Changes will be deployed to GitHub Pages when merged to main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the follow-up tracked in
docs/plans/resilient-automation.md:71-73.Why
PatchDeck could not say how much paid agent work it had done.
agent_runslooks like a spend ledger but is a per-babysit-session record — one row per session (babysitter.ts:3256-3282), written only bybabysitter.ts, withpr_id NOT NULL ... ON DELETE CASCADE. So issue work, CI healing, deployment healing, release notes, social posts, and PR Q&A were invisible, and PR-scoped history was destroyed with the PR.That made "should I leave Auto PRs and Auto Issues on overnight?" an open-ended question. This makes it a bounded one.
What
Ledger. New
agent_invocationstable: one row percodex/claudeprocess spawn, with duration, exit code, resolved agent, model, and outcome. Deliberately no foreign keys, so spend history survives deletion of the PR or issue that caused it.agent_runsis untouched.Attribution. Every paid path already funnelled through
runAgentCommand(agentRunner.ts:132), so the meter hooks in there. Each unit of work establishes anAsyncLocalStoragecontext viawithAgentWork(...); nested contexts win, so CI healing bills asheal_cirather thanbabysit_pr. A guard test pins the set of modules that reach an agent primitive, so a new call site cannot silently escape the meter.Ceiling.
maxAgentInvocationsPerHour, default 0 = unlimited — upgrading changes nothing. Two gates, because one is not enough:resolveClaimableKinds) stops claimingAGENT_INVOKING_JOB_KINDSwhen the ceiling is reached. Jobs stayqueued, exactly as under drain mode.babysit_prjob), the fallback-agent re-run atbabysitter.ts:4211, and conflict repair, which the dispatcher cannot see.The refusal classifies as
transientinfailureRecovery.ts, so the job backs off on the free cap instead of burning a paidmaxAgentRetryAttemptsslot — the agent never ran.The window rolls continuously;
resetsAtis when the oldest invocation ages out, not the top of the hour. Health-check probes are recorded but never counted, so opening Settings cannot exhaust a budget. Rows prune after 30 days on the existing retention sweep, and rows leftrunningby a hard shutdown are closed at boot.Surfaces.
GET /api/agent-spend, theget_agent_spendMCP tool, a header pill (only when a ceiling is set; amber at 80%, "paused until HH:MM" at the ceiling), and a Settings field beneath Max agent retry attempts.Tests
New coverage for each behaviour change:
agentSpend.test.ts— context nesting and unwind, ledger rows per spawn with duration/outcome, non-zero/timeout/thrown classification, refusal at the ceiling, rolling-window clearing, probes excluded, unattributed spawns still counted, and the call-site attribution guard.storage.test.ts— round-trip on both storages, window boundary, probe exclusion, orphan close, and a row survivingremovePR.backgroundJobDispatcher.test.ts— at the ceiling, agent kinds go unclaimed whilesync_watched_reposstill flows, the job staysqueuedwithattemptCount: 0, and raising the ceiling releases it.failureRecovery.test.ts— the refusal is transient, and survives losing its class across a serialization boundary.routes.test.ts,defaultConfig.test.ts,logsRetention.test.ts.Verification
npm run check,npx eslint .,npm run build,npm run test:all(827 tests) — all green.Plan and decision record:
docs/plans/agent-spend-metering.md.